Descriptive checks

Here we provide a descriptive overview of the syllable identifications relative to target (table 1). In the current data the number of syllables identified by EasyAlign perfectly matched the targeted number of syllables, i.e., in 100% of the trials there were 0 differences in the number of syllable detected versus target. Note however that we will manually check the EasyAllign syllable detections for the final dataset.

Table 1. Percentage of syllable detection mismatches (#tab:table01)
A summary table of percentage of differences between syllables
syllable differences percentage
0 100.00

Table 2 provides the percentages of different type of L2 stress placement matches and mismatches.

Table 2. Percentage of correct L2 placements (#tab:unnamed-chunk-1)
A summary table of percentage of stress match/mismatch types
stress mis/match type stress difference percentage
L2 correct same 33.93
L2 incorrect & L1 match same 0.00
L2 incorrect & L1 mismatch same 16.07
L2 correct difference 44.05
L2 incorrect & L1 match difference 0.00
L2 incorrect & L1 mismatch difference 5.95

Main Confirmatory Analysis

Gesture vs. no gesture

For the first analysis we simply assess whether the absolute difference in stress timing relative to the target stress time is different for the gesture or the no gesture condition.

D$accuracy <- abs(D$stressed_mistimingL2L1) #absolute deviation from stress from L2

#basemodel predicting the overall mean accuracy
model0 <- lme(accuracy~1, data = D, random = list(~1|ppn, ~1|target), method = "ML", na.action = na.exclude)

#alternative model with gesture versus no gesture as predictor
model1 <- lme(accuracy~condition, data = D, random =  list(~1|ppn, ~1|target), method = "ML", na.action = na.exclude)
anova(model0, model1) #test difference basemodel versus model 1
##        Model df      AIC      BIC    logLik   Test  L.Ratio p-value
## model0     1  4 4069.241 4084.509 -2030.620                        
## model1     2  5 4066.132 4085.217 -2028.066 1 vs 2 5.109432  0.0238
summary(model1) 
## Linear mixed-effects model fit by maximum likelihood
##  Data: D 
##        AIC      BIC    logLik
##   4066.132 4085.217 -2028.066
## 
## Random effects:
##  Formula: ~1 | ppn
##         (Intercept)
## StdDev:    0.011073
## 
##  Formula: ~1 | target %in% ppn
##         (Intercept) Residual
## StdDev:     58.9668 85.65074
## 
## Fixed effects: accuracy ~ condition 
##                       Value Std.Error  DF  t-value p-value
## (Intercept)        40.91071  8.046698 167 5.084162  0.0000
## conditionnogesture 21.28571  9.373199 167 2.270912  0.0244
##  Correlation: 
##                    (Intr)
## conditionnogesture -0.582
## 
## Standardized Within-Group Residuals:
##        Min         Q1        Med         Q3        Max 
## -1.7144678 -0.4332536 -0.1847360 -0.1847359  4.1253449 
## 
## Number of Observations: 336
## Number of Groups: 
##             ppn target %in% ppn 
##               2             168

It is of course possible that there is a contextual effect of gesture, depending on whether there is a stress difference or a presence of an accent (see Figure 2)

Figure 2.

We will further assess this in a complex model we expand our analysis with the relevant stimuli conditions, as well as their interactions with the gesture condition. If the interactions are statistically reliable we will perform a post-hoc comparisons with R-package “lsmeans” with a bonferroni correction.

#alternative model with gesture versus no gesture as predictor
model2 <- lme(accuracy~condition+stress+accent, data = D, random =  list(~1|ppn, ~1|target), method = "ML", na.action = na.exclude)
model3 <- lme(accuracy~condition+stress+accent+
                condition*stress+
                condition*accent, 
              data = D, random =  list(~1|ppn, ~1|target), method = "ML", na.action = na.exclude)


anova(model1, model2, model3) #test difference basemodel versus model 1
##        Model df      AIC      BIC    logLik   Test  L.Ratio p-value
## model1     1  5 4066.132 4085.217 -2028.066                        
## model2     2  7 4067.404 4094.124 -2026.702 1 vs 2 2.727168  0.2557
## model3     3  9 4070.265 4104.619 -2026.132 2 vs 3 1.139482  0.5657
#summary model 3 post hoc
sum3 <- summary(model3)
posthoc3a <- lsmeans(model3, list(pairwise ~ condition|stress),  adjust="bonferroni")
posthoc3b <- lsmeans(model3, list(pairwise ~ condition|accent),  adjust="bonferroni")

Click for model results

```r
sum3
```

```
## Linear mixed-effects model fit by maximum likelihood
##  Data: D 
##        AIC      BIC    logLik
##   4070.265 4104.619 -2026.132
## 
## Random effects:
##  Formula: ~1 | ppn
##         (Intercept)
## StdDev: 0.009251582
## 
##  Formula: ~1 | target %in% ppn
##         (Intercept) Residual
## StdDev:    58.19645 85.36076
## 
## Fixed effects: accuracy ~ condition + stress + accent + condition * stress +      condition * accent 
##                                             Value Std.Error  DF   t-value
## (Intercept)                              52.35119  13.93053 165  3.758017
## conditionnogesture                        9.94048  16.27766 165  0.610682
## stressdifference                        -21.29762  16.08559 164 -1.324018
## accentaccent present                     -1.58333  16.08559 164 -0.098432
## conditionnogesture:stressdifference       3.00000  18.79582 165  0.159610
## conditionnogesture:accentaccent present  19.69048  18.79582 165  1.047599
##                                         p-value
## (Intercept)                              0.0002
## conditionnogesture                       0.5423
## stressdifference                         0.1873
## accentaccent present                     0.9217
## conditionnogesture:stressdifference      0.8734
## conditionnogesture:accentaccent present  0.2964
##  Correlation: 
##                                         (Intr) cndtnn strssd accntp cndtn:
## conditionnogesture                      -0.584                            
## stressdifference                        -0.577  0.337                     
## accentaccent present                    -0.577  0.337  0.000              
## conditionnogesture:stressdifference      0.337 -0.577 -0.584  0.000       
## conditionnogesture:accentaccent present  0.337 -0.577  0.000 -0.584  0.000
## 
## Standardized Within-Group Residuals:
##         Min          Q1         Med          Q3         Max 
## -1.60812794 -0.46910650 -0.23332169 -0.08683539  4.28520679 
## 
## Number of Observations: 336
## Number of Groups: 
##             ppn target %in% ppn 
##               2             168
```

```r
posthoc3a
```

```
## $`lsmeans of condition | stress`
## stress = same:
##  condition lsmean   SE df lower.CL upper.CL
##  gesture     51.6 11.4  1    -93.0      196
##  nogesture   71.3 11.4  1    -73.2      216
## 
## stress = difference:
##  condition lsmean   SE df lower.CL upper.CL
##  gesture     30.3 11.4  1   -114.3      175
##  nogesture   53.0 11.4  1    -91.5      198
## 
## Results are averaged over the levels of: accent 
## Degrees-of-freedom method: containment 
## Confidence level used: 0.95 
## 
## $`pairwise differences of condition | stress`
## stress = same:
##  2                   estimate   SE  df t.ratio p.value
##  gesture - nogesture    -19.8 13.3 165 -1.489  0.1385 
## 
## stress = difference:
##  2                   estimate   SE  df t.ratio p.value
##  gesture - nogesture    -22.8 13.3 165 -1.714  0.0883 
## 
## Results are averaged over the levels of: accent 
## Degrees-of-freedom method: containment
```

```r
posthoc3b
```

```
## $`lsmeans of condition | accent`
## accent = no accent:
##  condition lsmean   SE df lower.CL upper.CL
##  gesture     41.7 11.4  1   -102.8      186
##  nogesture   53.1 11.4  1    -91.4      198
## 
## accent = accent present:
##  condition lsmean   SE df lower.CL upper.CL
##  gesture     40.1 11.4  1   -104.4      185
##  nogesture   71.2 11.4  1    -73.3      216
## 
## Results are averaged over the levels of: stress 
## Degrees-of-freedom method: containment 
## Confidence level used: 0.95 
## 
## $`pairwise differences of condition | accent`
## accent = no accent:
##  2                   estimate   SE  df t.ratio p.value
##  gesture - nogesture    -11.4 13.3 165 -0.861  0.3906 
## 
## accent = accent present:
##  2                   estimate   SE  df t.ratio p.value
##  gesture - nogesture    -31.1 13.3 165 -2.342  0.0204 
## 
## Results are averaged over the levels of: stress 
## Degrees-of-freedom method: containment
```

Gesture-speech asynchrony as a function of trial conditions

In the previous analyses we know whether speech prosody performance increases or decreases as a function of gesture, stress difference, and accentedness. A further question is whether gesture-speech synchrony is affected by stress difference and accentedness.

Figure 3.

It seems that synchrony between gesture and speech is best in the accented and stress match condition.

```
##        Model df      AIC      BIC    logLik   Test   L.Ratio p-value
## model0     1  4 2044.734 2057.230 -1018.367                         
## model1     2  6 2047.489 2066.233 -1017.745 1 vs 2 1.2451018  0.5366
## model2     3  7 2049.444 2071.312 -1017.722 2 vs 3 0.0448689  0.8322
```

Gesture-speech asynchrony and the directionality of error

Now we should know whether gesture-speech synchrony can be affected by trial condition that may complicate correct stress placement. If indeed gesture-speech synchrony is affected, we can wonder about how gesture and speech diverge when they are more asynchronous. Firstly assess whether gestures

#basemodel predicting the overall mean accuracy
model0 <- lme(asynchrony_L2L1~1, data = subD, random = list(~1|ppn, ~1|target), method = "ML", na.action = na.exclude)

#alternative model with gesture versus no gesture as predictor
model1 <- lme(asynchrony_L2L1~stress*correct, data = subD, random =  list(~1|ppn, ~1|target), method = "ML", na.action = na.exclude)
anova(model0, model1) #test difference basemodel versus model 1
##        Model df      AIC      BIC    logLik   Test  L.Ratio p-value
## model0     1  4 2133.622 2146.118 -1062.811                        
## model1     2  7 2126.648 2148.515 -1056.324 1 vs 2 12.97427  0.0047
summary(model1) 
## Linear mixed-effects model fit by maximum likelihood
##  Data: subD 
##        AIC      BIC    logLik
##   2126.648 2148.515 -1056.324
## 
## Random effects:
##  Formula: ~1 | ppn
##         (Intercept)
## StdDev: 0.007961735
## 
##  Formula: ~1 | target %in% ppn
##         (Intercept) Residual
## StdDev:    130.1349 2.115984
## 
## Fixed effects: asynchrony_L2L1 ~ stress * correct 
##                                                         Value Std.Error  DF
## (Intercept)                                          14.14516  16.72969 163
## stressdifference                                     30.74958  22.54347 163
## correctL2 incorrect & L1 mismatch                    49.44575  32.69010 163
## stressdifference:correctL2 incorrect & L1 mismatch -210.84048  58.87324 163
##                                                      t-value p-value
## (Intercept)                                         0.845512  0.3991
## stressdifference                                    1.364013  0.1744
## correctL2 incorrect & L1 mismatch                   1.512560  0.1323
## stressdifference:correctL2 incorrect & L1 mismatch -3.581262  0.0005
##  Correlation: 
##                                                    (Intr) strssd cLi&Lm
## stressdifference                                   -0.742              
## correctL2 incorrect & L1 mismatch                  -0.512  0.380       
## stressdifference:correctL2 incorrect & L1 mismatch  0.284 -0.383 -0.555
## 
## Standardized Within-Group Residuals:
##          Min           Q1          Med           Q3          Max 
## -0.063643564 -0.007597103 -0.002391494  0.008405780  0.050266435 
## 
## Number of Observations: 168
## Number of Groups: 
##             ppn target %in% ppn 
##               2             168

Power analysis

https://link.springer.com/article/10.3758/s13428-021-01546-0